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This study compares School Leaders Licensure Assessment (SLLA) sub-scores with 
principal interns’ self-assessment sub-scores (ISA) for a principal internship evaluation 
instrument in one educational leadership graduate program. The results of the study will 
be used to help establish the effectiveness of the current principal internship program, 
performance on the School Leaders Licensure Assessment, and provide suggestions for 
program improvement. The researcher compared the sub-scores for the multiple choice 
section of the SLLA with sub-scores from the university’s principal interns’ self- 
assessments to look for correlations between the two data sets. The results indicated a 
positive correlation in two areas of the study. 
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Introduction 

The use of data for program improvement is 
becoming commonplace not only in prek-12 
school systems, but also in university programs 
across the nation. No longer is it acceptable to 
simply teach graduate classes and assume the 
students have learned what they need to succeed 
in their fields of study; educational programs 
now make use of national standards when 
developing course content, and often these 
standards are linked to some form of national 
assessment. This is true of most educational 
leadership programs in the United States, as a 
part of accreditation requirements from national 


accrediting agencies (TEAC, 2012). Currently 
32 states require students seeking licensure in 
school administration to pass the School Leaders 
Licensure Assessment (SLLA), which is based 
on the Interstate School Leaders Licensure 
Consortium (ISLLC) standards (ETS, 2012). 
The ISLLC standards were designed to provide 
the framework for the preparation of school 
principals (Hessel and Holloway, 2002). 

Real program improvement does not happen 
if there is no accountability for the program 
learning outcomes. Knowing this, national and 
regional accrediting agencies such as the 
Council of Accreditation for Educator 
Preparation (CAEP) and the Southern 
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Association for Colleges and Schools (SACS) 
have mandates to use student learning outcomes 
as key elements in supporting program 
improvement initiatives. The challenge for 
educational leadership programs in the past has 
been a lack of useful data related to student 
outcomes on the SLLA exam. 

ETS has been providing universities with 
overall student scores on the SLLA for more 
than a decade. Though overall results can be 
informative to some extent, it provides 
leadership programs with very little useful data 
when it comes to program improvement. 
Luckily this form of reporting has recently 
changed for the better. Beginning in the 2009- 
10 testing cycle, ETS began reporting sub-scores 
on the SLLA to participating universities. These 
scores provide principal preparation programs 
with a wealth of new data to conduct research, 
particularly for those programs that claim the 
Interstate School Leaders Licensure Consortium 
(ISLLC) standards as their learning outcomes. 
The availability of SLLA Content Category 
scores provides a key database for the study of 
principal preparation program outcomes. 

Purpose 

This study looks at the relationship of five 
variables found in the SLLA compared to five 
variables on an independent performance 
indicator, the Internship Self-Assessment (ISA), 
for educational leadership graduate students at 
one university in the southeast United States. At 
the time of this study, very little research was 
found to assess the performance of principal 
internship programs as they relate to candidate 
sub-scores on the SLLA, and none could be 
located as it relates to student self-assessment 
scores. 

In this study, an analysis was conducted to 
compare student self-assessment scores for 
ISLLC Standards 1-5 received during the 
principal internship on the ISA, with student 
results from Content Categories I-V found in the 
SLLA. The results of the study will be used to 
help establish the effectiveness of the current 
principal preparation program at the university, 
provide suggestions for improvement, and 
recommendations for further study. 


Research Question: 

1. Is there a correlation between individual 
sub-scores 1-5 given by students on the 
Principal Internship Self-Assessment, 
and corresponding sub-scores on the 
School Leaders Licensure Assessment? 

Null Hypothesis: 

1. There is no correlation between 
individual sub-scores 1-5 given by 
students on the Principal Internship Self- 
Assessment, and corresponding sub¬ 
scores on the School Leaders Licensure 
Assessment. 

Definition of Terms 

Interstate School Leaders Licensure 
Consortium (ISLLC) Standards. The 

Interstate School Leaders Licensure Consortium 
(ISLLC) developed six standards for future 
school leaders where the focus is on student 
success (Council of Chief State School Officers, 
1996). Engler (2004) noted that the ISLLC 
standards are research-based, with the 
knowledge, dispositions, and performances 
necessary for exemplary school leadership. 
When taken as a whole, they are aligned with 
four broad themes: a vision for success, a focus 
on teaching and learning, an involvement of all 
stakeholders, and a demonstration of ethical 
behavior (Hessel & Holloway, 2002). The 
ISLLC standards have become the foundation 
for many principal preparation programs across 
the United States; at least 35 states use the 
ISLLC standards to guide policy and practice as 
it relates to principal preparation programs (Hale 
and Moorman, 2003). 

The six standards include: (1) the vision for 
learning; (2) the culture of teaching and 
learning; (3) the management of learning; (4) 
relationships with the broader community to 
foster learning; (5) integrity, fairness, and ethics 
in learning; and (6) the political, social, 
economic, legal, and cultural context of learning 
(Council of Chief State School Officers, 1996). 

Intern’s Self-Assessment (ISA) Sub-scores 
(Scores for Standards). The instrument for 
measuring student success in the internship for 
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this study is the Intern’s Self-Assessment (ISA) 
which was developed, validated, and tested for 
reliability by the faculty in the leadership 
program (Arroyo, Koonce, & Hanes, 2008 and 
Koonce & Kelly, 2012). The ISA is a survey 
which contains 24-items using a Likert-type 
scale instrument derived from the ISLLC 
standards. There are four items per standard 
(total six standards) with each item rated on a 0 
- 4 point scale ranging from “fails to address/no 
evidence of knowledge, understanding, and/or 
application” to “very specific/convincing 
evidence of knowledge, understanding, and/or 
application” (Hessel & Holloway, 2002, p.24). 
The items in the assessment came from the 
“Components of Professional Practice for 
School Leaders” (p.27). Each sub-score on the 
ISA is directly linked to a correlating ISLLC 
standard. 

School Leaders Licensure Assessment 
(SLLA). The School Leaders Licensure 
Assessment (SLLA) is the primary test used for 
granting principal certification and/or 
endorsement in over 32 states (Adams & 
Copeland, 2005). Ellet (1999) and Reese & 
Tannenbaum (1999) both report that the SLLA 
designers used the ISLLC standards to construct 
the SLLA instrument. The SLLA is used as a 
measure to determine if “entry-level educational 
leaders” have the knowledge necessary for their 
professional practice (Educational Testing 
Service, 2012, p. 1). The current assessment is 
divided into two sections, 100 multiple choice 
questions and 7 constructed response questions. 
The official score range for candidates on the 
SLLA is between 120 and 200 points. States 
utilizing the SLLA for licensure/endorsement set 
their own cut score (Educational Testing 
Service, 2012-2013). 

School Leaders Licensure Assessment 
Sub-scores (Content Categories I-VI). The 

SLLA is grouped into six categories with a 
separate score for each category. The six 
categories align to the six ISLLC standards: (1) 
vision and goals; (2) teaching and learning; (3) 
organizational systems and safety; (4) 
collaborating with key stakeholders; (5) ethics 
and integrity; and (6) education system 
(Tannenbaum & Robustelli, 2008). There are 
separate scores for the multiple-choice and 
constructed-response questions in two of the 


categories (Educational Testing Services, 2009). 
For testing and reporting puiposes, the SLLA is 
broken down into two sections; Section I 
consists of 100 multiple choice questions, and 
Section II contains seven constructed response 
questions. In Section I, the test covers material 
from ISLLC standards I-V. ISLLC standards I, 
II and VI are covered in the constructed 
response section of the assessment (Educational 
Testing Services, 2012-13, p. 11). This study 
utilized only the SLLA scores in Content 
Categories I-V because Section I of the SLLA 
(multiple choice responses) specifically aligns 
with ISA Standards 1-5. Constructed response 
questions (Section II of the SLLA) were not 
used because constructed responses are open to 
some level of subjectivity in grading. 

Literature Review 

Published research that compares SLLA 
results to principal preparation programs still 
remains scarce in the field of education. Though 
most educational leadership programs are 
required to track student progress and 
demonstrate effectiveness to acquire national 
accreditation, this data is rarely published in 
professional journals. Even less research is 
available regarding SLLA sub-scores as they 
relate to leadership program initiatives. There is 
minimal empirical evidence to document 
measures for correlating the SLLA sub-scores 
with another variable, particularly with a 
student’s self-assessment while in a leadership 
program, and no research could be found that 
looks at an intern’s self-assessment as it 
compares to his/her results on the SLLA. 

Kelly & Koonce (2012) did conduct a study 
related to overall SLLA results and intern self- 
assessments. The results found no correlation 
between SLLA overall scores and the overall 
scores on the interns’ self-assessments. The 
recommendations from this study determined a 
need to disaggregate the data further and 
recommended correlating sub-scores on the ISA 
with sub-scores on the SLLA. Koonce and Kelly 
(2013) also studied mentor’s assessment scores 
for interns derived from practicing principals in 
the field, and compared the sub-score results to 
sub-scores on the SLLA. The results indicated 
no correlations between the mentoring 
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principals’ scores during the internship 
experience and student sub-score results on the 
SLLA. 

The lack of studies related to principal 
internship preparation programs and sub-score 
results on the SLLA does not diminish the need 
for well-developed internship programs; there 
are numerous reports on the importance of 
quality field experiences that are based on the 
ISLLC standards (Southern Region Education 
Board, 2005; Southern Region Education Board 
2007; Davis, Darling-Hammond, LaPointe & 
Meyerson, 2005; Davis, Darling-Hammond, 
LaPointe & Meyerson , 2007; & Hernandez, 
Roberts & Menchaca, 2012). In order for these 
programs to be effective, they must take an 
honest look at student results, and make program 
decisions for improvement based on this data. 
That is the premise for this research. 

Methodology 

Participants 

A total of seventy-four (74) graduate 
students who completed the educational 
leadership program were studied in this research 
from a single university over a three year period. 
Not all program completers were included in the 
research sample, but it does include all program 
completers who successfully completed the 
principal internship, took the SLLA assessment 
at the conclusion of the program, and reported 
the scores back to the university. 
Approximately thirty students who were 
program completers in the date range of the 
study either did not complete the principal 
internship, or take the SLLA exam, and were 
therefore excluded from the dataset and not 
included in the study. 

Participants were not compensated in any 
form, and they were not interviewed, tested or 
surveyed beyond the normal program 
requirements. The participant group makes up a 
puiposeful sample consisting of all students that 
completed the internship program, and who also 
took the SLLA assessment between September 
2010 and May 2013. 


Instruments 

Two measures utilized in this study were 
scores from the five content categories (I-V) 
found in the SLLA and the six scoring standards 
(1-5) found in the ISA. Content categories for 
both instruments are directly aligned with the 
first five ISLLC standards. Tannenbaum & 
Robustelli (2008) established content validity for 
the most recent version of the SLLA through 
occupational credentialing and a job survey 
made up of practitioners and experts from the 
field. Performance standards were broken down 
for each of the sub-categories. Content validity 
for the ISA was established by Cannizzaro 
(2007). Sets of two raters used the ISA and 
discussed the outcomes in their focus groups. In 
addition, a rubric was in use that assured closer 
alignment with different scorers. Practitioners 
rated the ISA similarly. Subjective scoring 
(inter-rater-reliability/consistency between tests) 
is helped when usable guidelines for scoring are 
developed such as the use of the scoring rubric 
for the ISA (Koonce & Kelly, 2012). 

Data Collection and Analysis 

Data were collected using the SLLA sub¬ 
scores (Categories I-V) and the sub-scores on 
the ISA (Standards 1-5) as identified from the 
six ISLLC standards. Sub-scores were matched 
for the two measures, SLLA Category I with 
ISA Scoring Standard 1, followed by SLLA II 
with ISA 2 and the same matching with 
remaining Categories III-V and Scoring 
Standards 3-5. 

All data were reviewed for accuracy prior to 
inclusion in the study. Results for each student 
on the ISA and the SLLA were loaded into an 
excel spreadsheet. Any students who either did 
not have scores on the ISA, or sub-scores on the 
SLLA reported back to the university were 
removed from the data set. The names of 
students were then redacted from the 
spreadsheet and replaced with numeric coding as 
identifiers. The data set was then uploaded into 
the Statistical Package for the Social Sciences 
(SPSS) for analysis. 

To test the null-hypothesis that there is no 
correlation between the student SLLA sub¬ 
scores, and the student Principal Intern Self- 
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Assessment sub-scores, five separate bivariate 
correlations were run using Pearson’s r in the 
SPSS software package. Coladarci (et. al 2011) 
noted that Pearson’s r is “by far the most 
frequently used correlation coefficient in the 
behavioral sciences.” (p. 135). Significance was 
determined at the 0.05 level. If the Pearson’s r 
test revealed a difference significant at the 0.05 
level, the Bonferroni correction to a was 
implemented (Simon, 2008) to reduce the 
possibility of a Type 1 error. In addition, the 
researchers also reviewed the effect size to help 
analyze the level of correlation. Cohen (1988) 
defined effect sizes as "small, d = .2," "medium, 
d = .5," and "large, d =.8".Thus, a finding of .2 
would indicate a small correlation between the 
variables, while a finding of .8 or greater would 
indicate a very large or significant correlation 
between the two groups. 

The five correlations that were analyzed were as 
follows: 

1. SLLA sub-scores for ISLLC Standard 1 
to ISA sub-scores for ISLLC Standard I. 


2. SLLA sub-scores for ISLLC Standard 11 
to ISA sub-scores for ISLLC Standard 
II. 

3. SLLA sub-scores for ISLLC Standard 

III to ISA sub-scores for ISLLC 
Standard 111. 

4. SLLA sub-scores for ISLLC Standard 

IV to ISA sub-scores for ISLLC 
Standard IV. 

5. SLLA sub-scores for ISLLC Standard V 
to ISA sub-scores for ISLLC Standard 
V. 

Findings 

SLLA Sub-score I and ISA Sub-score I 

A Pearson’s r correlation coefficient was 
computed to assess the relationship between 
student SLLA sub-scores and sub-scores 
students assigned themselves on the Internship 
Self-Assessment (ISA) for ISLLC Standard I. 
Tables 1 and 2 provide the descriptive statistics 
and a detailed analysis from SPSS. 


Table 1 


Descriptive Statistics 



N 

Minimum 

Maximum 

Mean 

Std. Deviation 

SLLA 1 

74 

6 

14 

10.15 

1.878 

Mean 1 

74 

2 

4 

3.46 

.528 

Valid N (listwise) 

74 






Table 2 

Correlations 



SLLA 1 

Mean 1 

Pearson Correlation 

1 

.123 

SLLA 1 c . ,, 

Sig. (2-tailed) 


.297 

N 

74 

74 

Pearson Correlation 

.123 

1 

Mean 1 c . 

Sig. (2-tailed) 

.297 


N 

74 

74 
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The analysis determined that there is no 
correlation between the two variables, r = 0.297, 
n = 74, p = 0.123. The results indicate that there 
is no relationship between how students self- 
evaluate themselves for ISLLC Standard I on the 
ISA, and the students’ sub-score results on the 
SLLA for ISLLC Standard I. 


SLLA Sub-score II and ISA Sub-score II 

A Pearson’s r correlation coefficient was 
computed to assess the relationship between 
student SLLA sub-scores and sub-scores 
students assigned themselves on the Internship 
Self-Assessment (ISA) for ISLLC Standard II. 
Tables 3 and 4 provide the descriptive statistics 
and a detailed analysis from SPSS. 


Table 3 

Descriptive Statistics 



N 

Minimum 

Maximum 

Mean 

Std. Deviation 

SLLA 2 

74 

9 

20 

14.89 

2.615 

Mean 2 

74 

3 

4 

3.36 

.413 

Valid N (listwise) 

74 






Table 4 

Correlations 


1 

SLLA 2 

Mean 2 


Pearson Correlation 

1 

.626" 

SLLA 2 

Sig. (2-tailed) 


.000 


N 

74 

74 


Pearson Correlation 

.626" 

1 

Mean 2 

Sig. (2-tailed) 

.000 



N 

74 

74 


**. Correlation is significant at the 0.01 level (2-tailed). 


The analysis determined that there is a 
correlation between the two variables, r = 0.000, 
n = 74, p = 0.626. The results indicate that there 
is a positive correlation between how students 
self-evaluate themselves for ISLLC Standard II, 
and the students’ sub-score results on the SLLA 
for ISLLC Standard II. As the initial test 
revealed a level of significance at the 0.05 level, 
the Bonferroni adjustment was applied. Since 
the researchers were studying 5 separate 
correlations, the Bonferroni application results 
in using 0.01 as the level of significance. In the 
case of ISLLC Standard II, this level ultimately 
determined that there was a correlation between 
the two variables (r=0.000), with a medium to 


large (0.626**) initial effect size. The results 
indicate that there is a positive relationship 
between how students self-assess their 
performance during the internship experience for 
ISLLC Standard II, and the students’ sub-score 
results on the SLLA for ISLLC Standard II. 

SLLA Sub-score III and ISA Sub-score III 

Tables 5 and 6 provide the results from the 
Pearson’s r correlation coefficient computed to 
assess the relationship between student SLLA 
sub-scores and sub-scores students assigned 
themselves on the Internship Self-Assessment 
(ISA) for ISLLC Standard III. 
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Table 5 

Descriptive Statistics 



N 

Minimum 

Maximum 

Mean 

Std. Deviation 

SLLA 3 

74 

6 

12 

9.54 

1.563 

Mean 3 

74 

1.50 

4.00 

3.5282 

.53355 

Valid N (listwise) 

74 






Table 6 

Correlations 


1 

SLLA 3 

Mean 3 


Pearson Correlation 

1 

.017 

SLLA 3 

Sig. (2-tailed) 


.885 


N 

74 

74 


Pearson Correlation 

.017 

1 

Mean 3 

Sig. (2-tailed) 

.885 



N 

74 

74 


The analysis determined that there is no 
correlation between the two variables, r = 0.885, 
n = 74, p = 0.017. The results indicate that there 
is no relationship between how students self- 
evaluate themselves for ISLLC Standard 111 on 
the ISA, and the students’ sub-score results on 
the SLLA for ISLLC Standard 111. 


SLLA Sub-score IV and ISA Sub-score IV 

Pearson’s r correlation coefficient was again 
computed to assess the relationship between 
student SLLA sub-scores and sub-scores 
students assigned themselves on the Internship 
Self-Assessment (ISA) for ISLLC Standard IV. 
The results are displayed in tables 7 and 8. 


Table 7 


Descriptive Statistics 



N 

Minimum 

Maximum 

Mean 

Std. Deviation 

SLLA 4 

74 

7 

17 

13.19 

2.137 

Mean 4 

74 

1.5 

4.0 


.4168 

Valid N (listwise) 

74 






Table 8 

Correlations 


1 

SLLA 4 

Mean 4 


Pearson Correlation 

1 

.670** 

SLLA 4 

Sig. (2-tailed) 


.000 


N 

74 

74 


Pearson Correlation 

.670** 

1 

Mean 4 

Sig. (2-tailed) 

.000 



N 

74 

74 
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**. Correlation is significant at the 0.01 level (2-tailed). 

The analysis determined that there is a 
correlation between the two variables, r = 0.000, 
n = 74, p = 0.670. The results indicate that there 
is a positive correlation between how students 
self-evaluate themselves for ISLLC Standard IV, 
and the students’ sub-score results on the SLLA 
for ISLLC Standard IV. As the initial test 
revealed a level of significance at the 0.05 level, 
the Bonferroni adjustment was once again 
applied at the 0.01 level of significance. In the 
case of ISLLC Standard IV, this level ultimately 
determined that there was a correlation between 
the two variables (r=0.000), with a medium to 
large (0.670**) initial effect size. The results 
indicate that there is a positive relationship 


between how students self-assess their 
performance during the internship experience for 
ISLLC Standard IV, and the students’ sub-score 
results on the SLLA for ISLLC Standard IV. 

SLLA Sub-score V and ISA Sub-score V 

The final Pearson’s r correlation coefficient 
was computed to investigate the relationship 
between student SLLA sub-scores and sub¬ 
scores students assigned themselves on the 
Internship Self-Assessment (ISA) for ISLLC 
Standard V. The results are displayed in tables 9 
and 10. 


Table 9 


Descriptive Statistics 



N 

Minimum 

Maximum 

Mean 

Std. Deviation 

SLLA 5 

74 

8 

17 

12.32 

2.041 

Mean 5 

74 

2 

4 

3.62 

.489 

Valid N (listwise) 

74 






Table 10 

Correlations 


1 

SLLA 5 

Mean 5 


Pearson Correlation 

1 

-.122 

SLLA 5 

Sig. (2-tailed) 


.300 


N 

74 

74 


Pearson Correlation 

-.122 

1 

Mean 5 

Sig. (2-tailed) 

.300 



N 

74 

74 


The analysis determined that there is no 
correlation between the two variables, r = - 
0.122, n = 74, p = 0.300. The results indicate 
that there is no relationship between how 
students self-evaluate themselves for ISLLC 
Standard V on the ISA, and the students’ sub¬ 
score results on the SLLA for ISLLC Standard 
V. 


Conclusions and Recommendations 

This study was an attempt to address the 
lack of published research available from 
university educational leadership preparation 
programs that review their own assessment data. 
The researcher compared the sub-scores for the 
multiple choice section of the SLLA with sub¬ 
scores from the university’s internship self- 
assessment to look for correlations between the 
two data sets. The results indicated positive 
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correlations in two areas of the study. Both areas 
indicated a medium to large effect size, 
indicating a strong relationship. This indicates 
that the student self-assessments on ISLLC 2 
and 4 could potentially be used as predictors for 
student outcomes on the SLLA in those areas. 

Since additional relationships were not 
found in the other three areas of the SLLA, it is 
recommended that other quantitative measures 
be reviewed to study SLLA sub-scores and ISA 
sub-scores that may lead to program 
improvement. It is recommended the program 
faculty review the current rubric used by interns 
when completing the self-assessment to assure it 
clearly delineates what level of comprehension a 
student needs to demonstrate for each category 
and subsequent score. The hope that is tighter 
alignment to the rubric will result in stronger 
correlations between the SLLA and ISA scores. 

In addition, it is recommended that the 
program faculty conduct a review of current 
course objectives related to all of the classes that 
contain ISLLC standards to ensure alignment 
between what is written, taught and tested. A 
thorough review of the program course 
objectives has not been conducted since the 
change in the SLLA exam, and as such is needed 


to determine if there are any areas that are being 
missed in the curriculum. 

Finally, this study can serve as a foundation 
for other leadership programs to begin reviewing 
their own data in efforts of program 
improvement. The more research that is 
conducted in the area of principal internships 
and SLLA results, the better educational 
leadership programs will be across the nation. 
This in turn will lead to future administrators 
coming into the field more prepared for the 
careers in which they embark. 
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